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ABSTRACT 


This study consists of a factor analysis of the behav- 
ioral dimensions of the current Marine Corps Performance 
Evaluation System, using randomly selected fitness reports 
for officers in the grades Second Lieutenant through Colonel. 
The objectives and uses of performance eeannate on boeh ia 
the Marine Corps and in general, are discussed and the factor 
analysis methodology is developed. The fitness report data 
is then analyzed, leading to an identification of three 
magerilying factors common to all behavioral dimensions. The 
nature and composition of these factors is then discussed. 
Finally, it is concluded that the Marine Corps Fitness 
Report could be effectively simplified by using the three 
common factors rather than the 22 dimensions in current 


Se. 
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te LN CROLUCraon 


The purpose of this thesis is to conduct an analytic 
study of the Marine Corps' Performance Evaluation System to 
determine whether or not the system, as currently implemented, 
measures clearly, concisely, and unambiguously that which it 
is designed to measure. The study employs a statistical 
technique called factor analysis and relies on data supplied 
by Headquarters, United States Marine Corps (Code MMPA), 
facmington, D. C. 

Performance evaluation in general, and the Marine Corps 
system in particular, are discussed. The factor analysis 
Peenmigue is developed, not rigorously, but in sufficient 
detail to allow readers to understand the meaning and sig- 
nificance of the results. 

The data is then analyzed, showing the statistical 
correlations between the dimensions of the evaluation, lead- 
ing to a demonstration of how the system could be streamlined 
by utilizing the few underlying factors rather than the many 


G@imensions. 


IIT. PERFORMANCE EVALUATION 


A. GENERAL 


Mine success Or=failure of any organization, large or 
euemel, complex or Simple, military or civilian, is tied 
directly and inextricably to the performance of its personnel. 
This is not to say that other factors do not exist, or even 
that they are insignificant, but ultimately it is people-- 
human beings--who make the organization succeed or fail. It 
1s people who run the machines, program the computers, and 
make the important decisions. And it is people who determine 
the objectives of the organization, set its standards of per- 
formance and evaluate its personnel on the basis of those 
standards. 

The key to this important determinant of organizational 
output is an effective and efficient personnel performance 
evaluation system. It iS Such a system which will, in the 
long run, determine the "quality" of the organizational actors 
and thus of the organization, for it determines who will rise 
Eemeositions of leadership and authority, and who will be de- 
moted~-or even fired. "Quality" is used here in a relative 
sense, meaning "degree of excellence when compared to the or- 
ganizational standards." More specifically, the system 
(ideally) measures the actual and potential COmemmbutions 


of the individual to the success of the organization. 


All organizations use some type of performance evaluation 
system. People are always judging the performance of others-- 
peers, subordinates, superiors. The system may be either for- 
mal or informal and its essential elements will vary widely, 
depending on the objectives, size, complexity and general 
Seamer ure Of the Organization itself. Each organization in- 
ternally defines its own objectives, its own standards, and 
must, therefore, determine its own evaluation technique. By 
far the most common rating technique and the one used by the 
Marine Corps is the subjective rating scale in which several 
dimensions of an individual's performance are measured against 
a standard (below average, average, above average, etc.). 

The advantages of this technique are that it is clear and easy 
to use, but it is also subject to rater biases, leniency, 
SCenmeral tendency, and the halo effect (having the rating on 
one dimension affect all others either positively or nega- 
tively). 

Regardless of the technique used, however, the objectives 
of the performance evaluation system are generally the same. 
According to E. F. Huse (5), these objectives are: 

1. Meeting organizational objectives and improving per- 
formance--the subordinate needs to sow what his objectives 
are and to obtain feedback about his performance and areas 
for improvement; 

2. Proper salary administration--the manager needs to 
review the performance of his subordinates in order to recom- 


mend proper salary action; 





Beeeeeco! Jectimng and storing data for future administrative 
actions on promotions, transfers, and demotions or discharges; 

4. Identifying training needs-~-the organization needs to 
maintain accurate, current information about the strengths 
and weaknesses of its employees in order to develop timely 
and appropriate training programs; 

5. Improving the selection of new employees--if an organi- 
zation is to have a systematized, valid selection system, it 
needs proper criteria against which to validate its selection 
instruments. 

A different technique or appraisal approach may be required 
for each of these objectives, but more often than not, organi- 
zations attempt to use an all-purpose form such as the sub- 
jective rating scale mentioned above to simultaneously satisfy 
feo: them, This 1s not to imply a criticism of either the 
all-purpose approach or the organization opting to use it. 

The all-purpose form may satisfy the objectives of the organi- 
zation to such an extent that the additional complexity of 
multiple-forms is not justified, or it may simply be an 


economic necessity. 


B. PERFORMANCE EVALUATION IN THE MARINE CORPS 


The objective of the Marine Corps performance evaluation 
system, as stated by Marine Corps Orders of the 1610.7-series 
ws EO provide, for all officers and noncommissioned officers, 

a fitness report which is an evaluation of duties performed and 


the manner in which performed. Such reports represent a com- 
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prehensive portrayal of the professional qualifications, 
personal traits and characteristics, and potential of the 
individual. The order further states that fitness reports are 
the principal record of a Marine's performance and conduct and 
as such are vital in determining duty assignments and in sel- 
ection for promotion. The system, as thus described by the 
order, embraces three of Huse's five objectives, omitting the 
objectives of providing records for training and selection of 
new employees. The requirements for such records obviously 
exists within the Marine Corps, but they are satisfied by 
other means. The principal instrument of the Marine Corps 
system 1S a single, all-purpose subjective rating form, an 
example of which iS contained in Appendix A. The form pro- 
vides for the measurement of 21 dimensions of performance on 
a rating scale from unsatisfactory to outstanding. There is 
als® a provision for a rating of "Not Observed" in case a 
particular performance dimension is not demonstrated during 
the period of the report. 

The Marine Corps Order defines 14 of the 21 performance 
Gimensions. They are: 

meeenaurance: “Physical and mental ability to carry on 
Meaer fatiguing conditions. 

yee eer cOnal Appearance:) The trazt of habitually appear—- 
mg neat, smart, and well-groomed in uniform or in civilian 
attire. 

3. Military Presence: The quality Gf MalnealningadlpEe= 


priate dignity and soldierly bearing. 


Jet 





4. Attention to Duty: Industry, Ne weal VOr  -WOrmka ng 
thoroughly and conscientiously. 

beeeoperateion:. Ihe faculty of workingg@an harmony) with 
others, both military and civilian. 

Pe ietaAtivew Fhe Eralt Of taking necessary Or appro- 
priate action on own responsibility. 

feeeeoudgment: The ability to think cllearly and arrive 
Eeeeetogical conclusions. 

See resence OL giind: The ability to think and act promptly 
and effectively in an unexpected emergency or under great 
strain. 

Meeeeorce: The faculty of carrying out with energy and 
resolution that which is believed to be reasonable, right 
ee, Guty . 

Doeeemeadership: The capacity to dissect, control, and 
influence others and still maintain high morale. 

ites Loyalty: The idWality ©£ rendesing faithful andgwilling 
service, and unswerving allegiance under any and all circum- 
Seances . 

12. Personal Relations: Faculty for establishing and main- 
taining cordial relations with both military and civilian 
associates. 

13. Economy in Management: Effective utilization of men, 
money, and materials. 

14. Growth potential: The capacity for professional devel- 


opment. 


jhe remaining / of the 21 dimensions are undefined and left 


ee 





fomtne interpretation of the “ater. Implicit in this omission 
is the assumption that they are self-explanatory, an assump- 
tion which may or may not be justified. These undefined 


Samensions are: 


1. Regular duties 

2. Additional duties 

3. Administrative duties 

4. Handling officers 

Se Handling snilaisted Personnel 
6. Training Personnel 


foe tacercal Handling of “Froops 


The report provides for the appraisal of a final dimen- 
sion, entitled "General Value to the Service". This item is 
the last evaluation on the fitness report and there is a temp- 
maeton, CONnSCiously or otherwise, for the raters to treat it 
aS a composite or average of the previous 21 marks. The 
Marine Corps Order, however, clearly and specifically advises 
against this, stating that it should be an estimate of how the 
Marine compares with all other Marines of the same grade (rank) 
Known to the rater, taking into consideration all important 
factors. It is thus intended to be a separate and unique 
mark, not a composite. 

All dimensions except the last (General Value to the 
Service) are rated on a six-point scale from unsatisfactory 
to outstanding. The scale values, with definitions from the 


order, are as follows: 
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1. Outstanding: one of the clearly superior individuals 
eienis grade known to the reporting officer (rater). 

2. Excellent: qualified to a degree seldom achieved by 
others Sf shwsmoguace: 

3. Above Average: highly qualified. 

4. Average: qualified to a generally accepted standard. 

See (Overs e Gage moc Low ene generally accep tcearstanidard . 


6. Unsatisfactory: unacceptable performance. 


The final dimension is rated on a ten~point scale, using 
the six scale values defined above plus intermediate values 
among the top five, i.e., between below average and average, 
between average and above average, etc.. 

A careful examination of the definitions of the dimen- 
Sons of the Marine Corps fitness report reveals a potential 
for overlap or confusion among them. For example, the defini-~ 
tion of initiative implies a certain amount of judgment, Since 
Mmemspecities “appropriate action". Similarly, it is difficult 
to imagine good personal relations in the absence of a positive 
Spirit of cooperation. To the developers of the form, each 
dimension undoubtedly applies to a separate and distinct aspect 
of performance, but what about to the users? Are they able 
to make the subtle distinctions or are they, in fact, evaluating 
a smaller number of dimensions by basing their evaluations of 
several dimensions on a single characteristic as they see or 
are mentally able to define it. If overlap actually does exist, 
the question arises as to whether or not the number of dimen- 


Sions could be reduced to the point where users are able to 
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clearly distinguish among them. Simplicity is certainly a 
worthy objective of any rating system, and if the form could 
be simplified to a smaller number of dimensions which the 
users see as distinct and unambiguous, while still providing 
an adequate evaluation of the relevant dimensions of perfor- 
mance, then such a step would necessarily be an improvement. 
To test the hypothesis that simplification is possible, 
that the essential characteristics of individual performance 
could, in fact, be evaluated by using a smaller number of 
dimensions, a large body of summary fitness report data was 


subjected to an analytic technique called factor analysis. 





itt. FACTOR ANALYoLs 


This chapter presents background information concerning 
factor analysis and discusses the methodology used in this 
study. Factor analysis is a statistical technique formulated 
Peepsychologists at the turn of this century to provide 
mathematical models for the development of psychological 
theories of human ability and behavior. Because of its 
Origin and extensive use in psychology, it is often regarded 
as psychological theory, but it has been adapted for use in 
other areas where numerous interacting measurements are ob- 
tained. Its use has greatly expanded as a consequence of 
the development of high-speed electronic computers. 

Since the objective of this study is to subject a set 
of Marine Corps fitness report data to factor analysis, 
rather than to either develop or illustrate factor analysis 
itself, the technique will not be described in great detail. 
What will be presented is just enough for the reader to be 
fees tO follow and understand the origin and nature of the 
answers and conclusions which result. For a more thorougn 
discussion of the technique, the reader is directed to any 
of the references listed in the bibliography, especially the 
@ae by H. H. Harmon. 

It should also be noted at this point that a Biomedical 
(BIMED) computer program (BMD08M) was nse in this analysis, 


so the computational details are not required to either 
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achieve or understand the results. A general understanding 


of the technique is, however, essential. 


Poe OBJECTIVES OF FACTOR ANALYSIS 


hem paimamy OCbgective @f factor analysis iS to obtain 
a parSimonious description of observed data. Harmon (4) sees 
it as a technique to resolve a set of variables into a small 
number of elements, called factors. Resolution is accom- 
plished by the analysis of the correlations between the var- 
jables. Factor analysis, then, is essentially a linear re- 
gression of each of the variables on the factors. It yields 
factors which provide an adequate fit to the data, while 
maintaining all the essential information of the original set 


@eevarilables. 


Pee LHE FACTOR ANALYSIS MODEL 


It is the object of factor analysis to represent a var- 
iable Ve in terms of several underlying factors, or hypo- 
thetical constructs. Several types of factors may be dis- 
EEnguished: 

mee  COmnOon Lactors 
acon era ter aClOmsmmpresent in all variables. 
DrmGEOusiadctor: Sisesent an more than one, but not 
al, variables. 
2. Unique Factors: present in only a single variable. 


Memmnon f£ackons account for the intercorrelations among the 


i 





variables, while each unique factor represents that portion 
Sea Variable not attributable to its correlations with 
other variables of the set. 

The simplest mathematical model for describing one var- 
jable in terms of several others is a linear one, and that 
is the form of representation used in factor analysis models. 
Using the notation Foy Fos Fo, coe, Fa =£Or mM COMMOnm Lactors, 


the complete linear expression for any variable We may be 


Wiemecen in the form 


ee ee oe oo ae 
where i= 1, 2, *** , n and B44 is the coefficient of the 
yen factor Of “Ene oe Variaole. Lhere are, Of course; on 


equations of this form--one for each of the n variables. 

Some models also include a term a.U, which denotes the 
Hoeaude aspect of any variable--i.e., that portion of its var- 
mamee which is not attributable to any common factor. Since 
meGtor analysis in general, and this paper in particular, is 
concerned primarily with the common factors, the unique term 


will not be included in the model used herein. 


C. FACTOR LOADING AND COMMUNALITY 


The coefficients a5 in equation (3-1), also called 
maeeor loadings, Can be determined through an analysis of 


the correlations among the n variables. All m factors are 


18 





required to reproduce the correlation among the original n 
variables, and each factor, through its loading, is selected 
to make maximum contribution to the sum of the variances of 
the original variables. The first such factor selected makes 
the greatest single contribution; the second makes a maximum 
Sonecibution to the remaining variance, and so on until a 
Satisfactory portion (usually less than 100%) of the total 
Original variance has been accounted for. Thus, depending 
on the amount of variance which will give a satisfactory and 
acceptable solution, only a small number (less than n) of 
factors will be needed to reproduce the original data. 

For any particular variable, the amount of its total 
variance accounted for by the common factors is called its 
Senaunality. Quantitatively, the communality of a variable 
1s given by the sum of the squares of the common-factor 


S@enricients, viz. 


De De 2 2 D 
(3~2) hi Sao ee as 5 + asa + tas. 
— ; sila 
where he isnenescommunmality Of the 1 variable V; and 
the ais mer les Tactoe COCTIICients. 


The residual variance (one minus the communality) is 
mie extent to which the variable 1S unique. It should be 
noted that although the communality can be increased by 
Simply increasing the number of. common factors extracted 


from the set of variables, this is not, in general, desirable. 
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Parsimonious description of the data requires that the num- 
mem.of factors be kept to a minimum. 

Factor analysis techniques require communality est- 
imates as inputs. Successive iteration then leads to the 
final correct communality values. Making the original est- 
imate, however, can sometimes pose a difficult problem. 


There are three principal and commonly used estimating tech- 


niques. They are: 
1. Set the original communality estimate equal to one 
£Or all variables-~i.e., assume that all of the variance of 


the variables will be accounted for by the factors selected. 

2. Use the squared multiple correlations as the commu- 
nality estimates. 

3. Use the maximum row values of the correlation matrix 
as the communality estimate. 

These three techniques are discussed in detail by Harmon 
(4) and others and each has considerable merit in a variety 
of circumstances. The first technique listed above was used 
memciis study, not because of any intrinsic superiority, 
but simply because of the author's subjective judgment. 

Having determined the communalities, it is then possible 
Memealculate the factor coefficients, or loadings. The most 
frequently used technique (principal-component) begins by 
choosing a set of factors in decreasing order of their con- 
tribution to the total communality. The analysis is begun 
Beeextracting a factor F whose Contribution to the commu- 


1. 


nalities of the variables is as great as possible. Then the 
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mms —LaAactor Gesiaual correlations are obtained. A second 


mactcor FP. with a maximum contribution to the residual 


communality is next found. This process 1S continued until 
the total communality has been analyzed. 
ieee recstoaractor coerrier ents S59) are selected to max- 


mezer the sum of the Contributions of that factor to the total 


eommunality. For the first factor (Fi), this sum is given by 


= ae pene’ cael 


(3-3) S, = el 31 nl’ 


Mine coefficients a. in equation (3-3) must be chosen so 


il 
as to maximize Ci under the constraint (for m factors) 
™m 
(3-4) 1S a » oie Cie a 25 o>. Stee ene 
Dau 
mere r.. = ry. and r.. is the communality of variable 
ats ea Li 
Vv. fi 5e. , lee Rec The constraint condition (3~4) says 


Pier the reproduced Correlations are to be replaced by the 
@bserved correlations, implying the assumption of no unique 
Variance (1.e., zero residual error). Recall that it was 
Carlier stated eek the unique factor would not be included 
in this model. 

Maximization of this function (3-3) of n variables, 
constrained by sn (n+1) COndtenoncms4), 25 greatly faci11— 


tated by the method of Lagrange multipliers, which may be 


applied as follows: define the Lagrangian function (L) such 


that 
ig) Jal m 
(3-5) 2L=C, - » Manta = Sa = » Miktip?kp 
i,k=1 i,k=l p=1 


where the Wi, ( = Uys) sive Elva Leeseciee mbllenjeliess,  Wavetn: 


set the partial derivative of L with respect to any one of 


the n variables (ay watch 2 = 1, 2, *-* |, m) equal te zero: 
n 
6) Ol _ 
ee Mix tkd1 oe 
sel 
k=l 


Similarly, set the partial derivative of L with respect to 


any of the other coefficients (a. for p * 1) equal to Zexe. 


p 


n 
aan a 
(3-7) aes ls ) Mixekp 
ip 


k=l 


The two sets of equations, (3-6) and (3-7), may be combined 


as follows: 


ae ee is 
ip 





mmere p= 1, 2, *** , m and the Kronecker Delta, aah = ] 


p= 1 and ¢ = 0 if p#1l. Multiplying equation 


(3-8) by aay and summing with respect to i leads to: 


nN Nn 16] 
Z - 
——” Mes » ain ~ % ron Se 
iI) lc 
renee, according to equation (3-6), 
nN 
2 a » Vix 
TIL 


then, by letting 


nN 
- 2 
Ma » an 
a 


where X is used for convenience only at this point but will 


be defined more rigorously later, equation (3-9) becomes 


(3-10) Sip) - 21 A2kp = 0 


n 
=] 


k 
Meet p lying, by ain and summing with respect to p, equation 


Zo 





(3-10) becomes 


i n 
(3-11) asi, = » #1 Y aptip SOs 
el 


p=l 


Peplying the constraining conditions (3-4) to equation (3-11), 


jee result is 


n 

(3-12) ) Tea - A434 = 0 el Aa Senin) 
k=1 

which is a system of n equations. Recalling that rie hé 

Becearopping the subscript of hy for convenience, this sys- 


tem of n equations may be written as follows: 


5 
(hy - A)ay, + ee : 
2 
T5793, + MQ ~ Adan, Free + Aen 0 
(13) ; : ‘ : Beis ; ; , é 
Yr ia aa a echo & te =e = ee 
nl1?11 a OT n fn 7 


The existence of a nontrivial solution to this system of 
equations requires that the determinant of the coefficients 


manish, as £Lollows: 


Z 
hy =} Yi5 eee Lin 
Z 
eonl No -_ r eo 8 Con 
ie 14) ® e @ e eee ‘ ° 7 0 
2p 
nl a mon 
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Expansion of this determinant results in an ia order 


polynomial in 4d’, known as the characteristic equation (of 
the system). The polynomial has a family of solutions, all 
Of which are proportional to one particular solution, with 


the factor of proportionality given by 


n 
_ 2 
hy = ) aie 
a 


Recall, however, from equation (3-3), that this is precisely 
Ener quantity to be maximized. It follows, therefore, that 


the maximizing solution to Cy 1s the largest root of the 


characteristic equation. To find the coefficients of the 
imeeot L£actor (F,) which will account for the maximum amount 


of communality, the value of hy 1S now substituted into the 


set of equations (3-13) and any solution O7q7 Gop 8% r By 


is obtained. To satisfy the conditions of equation (3-3), 
these values are divided by the square root of the sum of 


their squares and multiplied by VAY . the resultinge quan. 


mmeres are the desired coefficients of Fy in tie fac tes 


eemeern (3-1): 


a, VA 
(3-15) oa = ee 
i 2 2 i 
ool = O57 = eae a a al 
where i= 1, 2, *** , n. In the literature of mathematics, 


the roots (A's) of the characteristic equation are called 


eigenvalues, while the solution to the set of equations (3-14) 


Z> 





corresponding to each eigenvalue leads to a vector (a set of 
a's) which is called an eigenvector. 

The problem now remains to find the coefficients of the 
remaining factors, accounting for a maximum amount of the 
residual communality. The residual correlation after the 


first factor has been extracted is given by 
(3-16) rd = Yr 


ik ~ @41°k1 


and the guantity to be maximized is 


7 Z 2: 2 
(3a~17) Cy = Ain + aby + a2 
pleyect to the constraint condition of (3-16). Iteration of 
the method of Lagrange multipliers yields hos the second 
largest eigenvalue, as the maximizing value of Cy. The 


second-factor coefficients are then determined as above. 
Successive iteration of this procedure will eventually pro- 


duce the complete set of factor coefficients, or loadings. 


D. FACTOR ROTATION 


Once a set of factor Heacimoags Mas been calculaued,. tne 
next step in the analysis is to interpret the factors in a 
way that will give a meaningful summary of the observed data. 
Since the factor loadings are produced in an arbitrary frame 


of reference, the problem is to choose a reference frame for 
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the factor loading points which will give the most meaning- 
mol anayor most useful interpretation. To this end, the arbi- 
trary frame of reference may be rotated to one more suited to 
interpretation. There are numerous rotational techniques and 
criteria to select from. Thurstone (10), for example, has 
specified his criteria for simple structure which ideally 
weutad result in a relatively unigue configuration of factor 
loadings and a relatively standard location for the reference 
frame. As pointed out by Morrison (8), however, the problem 
with these criteria is that they rarely exist when using real, 
data. For simplicity, rotational techniques can be grouped 
into two broad classes: orthogonal and oblique. Although 
Smenegonal rotation is not suitable for all data, it has at 
least one distinct advantage. Since the resulting factors 

are orthogonal, they are uncorrelated (independent), which 
greatly facilitates interpretation. It should also be pointed 
out that the varimax orthogonal rotation technique was devel- 
oped by Kaiser (6) in 1958 to allow actual data to meet 
Thurstone's simple structure criteria as closely as possible. 


This is the rotational technique which was used in this study. 


Be FPACTOR SCORES 


From a purely theoretical point of view, the common 
factors have a more fundamental importance than the observed 
variables themselves and it is therefore necessary to relate 
the observations to the common factors. This is done by 


means of factor scores, which are a means of. expressing 
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quantitatively the information contained in a factor for a 
specific case or individual. Through the factor scores, the 
difference between two cases can be expressed in terms of the 
reproduced correlations of the original data. 

The computation of factor scores is based on the factor 
loadings previously determined. When using ones on the main 
Gmegonal Of the correlation matrix, aS in this study, the 
principal-factor solution may be expressed in matrix notation 


as follows: 


(2-15) V = AF 
where 
vew— mex 1 columm vector of variables, 
A = n Xn matrix of factor loadings, and 


Ey = nex 1 colgimm vector of factor scores. 
The factor scores are then given by: 


19) F = AV. 


Pee lL ACTOR INTERPRETATION 


Pmt eNom rackOr lOaGings and factor scores have been 
determined, there remains only the task of interpretation. 
A complete solution requires an identification of the nature 
fmemecCOntent, Of Lhe hyoothetical factors. Fruchter (3) in- 
dicates that this is commonly aqone by inferring what the var- 
jables with high loadings on a factor have in common that is 


present to a lesser degree in variables with moderate or low 
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loadings and absent from variables with zero (or near zero) 
loadings. He further defines an arbitrary classification 
scheme for factor loadings, as follows: 

ieeinsologntticant: Eactor loading below 0.2 

hoy se cactor loading @©f 0272 bo 0.3 

Be @dcmacow [actor loadtmg Of 0.3 tert.s 

eis cactor loading of 075 ford: 7 

ERemvery litgie factor loading above 0.7. 
Peeeeough Fruchter's classification scheme is admittedly arbi- 
trary, this phase of the analysis is necessarily less objec- 
tive and more subject to the desires and experience of the 
analyst than that which preceded. Even so, there is quan- 
fieative Justification for at least a portion of his classi- 
fication scheme. Recall that in linear regression, the 
Sagere Of the correlation coefficient indicates the propor- 
tion of the total variance explained by the regression. Thus, 
emeactor loading of 0.7, which separates the "high" and “very 
Peeimeeclassifications, corresponds to a level of correlation 
between the variable and factor in which nearly one half of 
the observed variance has been accounted for. 

A factor loading of 0.7 will therefore be adopted in 
meres tudy as being indicative of a “Significant” correlation 
between variable and factor. From there, an attempt will be 
made to assign convenient, understandable, and unambiguous 
names to the factors, based on an observation of those var- 


tables with significant loadings. 
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The data for this study consisted of the fitness report 
summaries from 450 randomly selected Master Brief Sheets 
(form NAVMC HQ 466) for Marine Corps Officers in the grades 
Second Lieutenant through Colonel. The data was furnished by 
Headquarters, U. S. Marine Corps (Code MMPA) upon my official 
written request. Each Master Brief sheet represents a com- 
plete fitness report record for a particular individual since 
mid-1972, when the Performance Evaluation System was automated. 
It contains all of the evaluations for the 22 dimensions pre- 
viously discussed. The personal information from Section A 
of the fitness report (see Appendix A) had been deleted from 
each Brief Sheet to insure the confidentiality of each indi- 
waemal's record. 

Of the original 450 Master Brief Sheets, 46 were found 
Pemcontain insufficient information to be useful in this 
fedeay. Additionally, four of the 22 dimensions were dominated 
by "Not Observed" marks and were therefore eliminated from 
analysis. The eliminated dimensions, with percentage of "Not 
Observed" marks, were as follows: 


1. Additional Duties - 893 


2. Tactical Handling —- 87% 
3. Endurance - 623 
4. Presence of Mind - 933 
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Since each Brief Sheet represents a separate observation of 
the variables (dimensions of performance), the data for the 
analysis consisted of 404 observations of 18 variables. 

Since the factor analysis technique is essentially 
mathematical in nature, it was necessary to convert the sub- 
jective non-numeric scale (excellent, above average, etc.) 
used by the reporting officers to a numeric scale suitable 
for the analytic technique. To do this it was first neces- 
Sary to make an assumption about the distribution of the 
attributes which the fitness report is designed to measure. 
The Normal, or Gaussian, Distribution was assumed for this 
study, aS 1S common in such cases. The argument for use of 
the Normal Distribution is strengthened by the fact that 
Pmmeametic means Of variable markings were eventually used 
in the analysis. By the Central Limit Theorem, these means 
tend to be normally distributed, regardless of the actual 
Meeertying distribution (9). 

In a random sample of 100 fitness reports (from the 404 
Master Brief Sheets), the frequency, relative frequency, 
cumulative relative frequency, and standard normal deviate 
(g=-vealtie) for each mark, or scale value (excellent, etc.) 
were determined. The results are shown in Table I. 

To calculate the mean, -. eee Veluwe. OL Cacn Om 
these marks (denoted by Or let £(X) represent the dis- 
mebeucion function of the marks within the interval [Z, 25], 
where Z4 and 2z are the minimum and maximum standard 


2 


normal deviates, respectively, of the mark. The expected 
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value is then given by: 


aD 
/ 2 (Oc eb 
Me 


(4-1) xX = 


E(x) ax 
at 


Since the normal distribution has been assumed, f£f£(X) is 


given by: 


(2) f(X) = ——_e (-0 < x < oo), 


After substitution and integration, the desired scale values 


are given by: 





: — _. 
(Ges) X es: 


F(z.) ~ F(z,) 


where the numerator can be determined by direct substitution 
of the z-values and the denominator can be determined from 
any table of areas under the normal curve. 

As an illustrative example, let denote the value 


@eean Unsatisfactory mark. Then 


=}. 5 (asl eee = es) 
S = e 





Var 


<I 
il 


un 
Eon = EH) 


which, after substitution and reduction becomes 


(Or 959 | OOO so 0) 


a MG = 0. 


Ton 264. 


eamilarly, ae the value of a Below Average mark is given 


by : 


: | 26 8) = Eos. 2272 
= 


a= 


x a V 27 


ba 





— & 
i (ak CS NS Ek | 


- (0.3989) (0.0145 - 0.0056) 
(0.0018 - 6.0006) 


mi 2 VO ee 


The results of these calculations were then subjected 
to a linear transformation to eliminate negative signs and 


decimal points. The transformation was of the form 
(4-4) p= 1000 (xX +47) 


where X is the scale value obtained from equation (4-3) and 
T ais the more convenient transformed value. The results of 
fmese Calculations and transformations are shown in Table II. 
The transformed values (T) were then substituted for 
their corresponding markings on each of the fitness reports 
and a statistical average for each of the 18 variables was 
obtained for each of the 404 individuals comprising the data 


base. The average values for each of the individuals, rather 
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Table II. The Numeric Marking Scale 


ORIGINAL TRANSFORMED 
MARKS VALUE VALUE 

(X) | (2) 
Unsatisfactory = a0) 204 02773 
Below Average a oe 1045 
Average = 2.655 1344 
Above Average = Ih ts) sia) 2142 
Excellent = We S127) 5) 3412 
Outstanding Cs) Gis) 7 4839 
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than the value of each mark on each of the available fitness 
reports, was used in the factor analysis to minimize the 
effect of "random error" caused by improper marking and rater 
bias. The result is the 404 by 18 matrix of markings con- 
tained in Appendix B. 

Peedi tene ro quantifying the imdividual markings, it 
was necessary to assign numbers to the variables to be 
analyzed, a somewhat more simple task. The numbering scheme 


mmshown in Table IIil. 
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felon tiie 


VARIABLE 
NUMBER 


10 
JL aE 
ie 
13 
14 
AS 
16 
ey 


18 


Variables to be Analyzed 


BEHAVIORAL 
DIMENSION 
Regular Duties 
Administrative Duties 
Handling Officers 
Handling Enlisted 
Training Personnel 
Personal Appearance 
Military Presence 
Attention to Duty 
Cooperation 
Initiative 
Judgment 
HOUuCce 
Leadership 
Loyalty 
Personal Relations 
Economy in Management 
Growth Potential 


General Value to the Service 


oy 





V. THE ANALYSIS 


Analysis of the data from the 404 observations of the 
18 variables was accomplished in two major steps. The first 
step utilized the complete data set, treating the sample 
population as an integral whole. In the second step, the 
data was separated into subsets, one subset for each rank 
(Second Lieutenant through Colonel) and one subset each for 
Company Grade Officers (Second Lieutenant, First Lieutenant, 
and Captain) and Field Grade Officers (Major, Lieutenant 
Colonel, and Colonel). The second step constituted an effort 
to determine whether or not significant observable differ- 
ences in factor structure exist between the various subgroups. 
In both steps, the data was analyzed using the Biomedical 
Factor Analysis program BMD08M. The type of rotation and the 
number of factors extracted were varied in an attempt to 
a@eount for ice ae ianec of a maximum number of variables. 
After a number of trial runs of the program, it was deter- 
mined that varimax orthogonal rotation for three factors pro- 
duced very high factor loadings (greater than 0.7) on 12 of 
the 18 variables and accounted for 73.5% of their total var- 
jance. Tne next best combination of number of factors and 
Pemaclon technique produced very high factor loadings on only 
10 variables, without an increase in the proportion of total 


Variance accounted for. The three-factor varimax orthogonal 
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rotation procedure was therefore selected for use in the 


remainder of the analysis. 


Pee ANALYSIS OF THE COMPLETE DATA SET 


Since factor analysis is essentially an analysis of the 
correlations between variables, a logical starting point is 
an examination of the correlation matrix, reproduced in Table 
IV. This table shows the correlations between all possible 
pairs of variables. It shows a wide range of correlations, 
memo. sl to 0.84, and eee Chiat. there. isssee tl facie mea 
high correlation between some of the variables. It does 
little, however, to illuminate a pattern which might reveal 
Ea@emunderiying factors. The most essential and useful infor- 
mation is contained in the matrix of factor loadings, shown 
in Table V. 

Using the criterion, previously established, of very 
high factor loadings being significant, Table V shows that 
the following variables have significant loadings on the 
mest factor: 

1. Regular Duties (variable 1) 

fee ea oa Personne Piveriable 5) 

Bee lnitiative ene 10) 

4, Judgment (variable 11) 

5. Force (variable 12) 

6. Leadership (variable 13) 

7. Economy in Management (variable 16) 


This means that these seven variables are all closely related 
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‘Weve yes We 
VARIABLE 1 
i 0.7435 
2 056749 
3 0.5836 
4 05975 
5 Oe 267 
6 Op) 31.9 
7 0.4413 
8 OFIGGIGZ 
S, Oo s05 
10 07 O70 
al 07270 
1a 0.8046 
13 OF FALSE 
14 0.2664 
d fee 052922 
KG O22 09 
7 0.5344 
Ag Oma & 
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PACTOR 


2 
Ore cioek 
0.1574 
Oi OOS 
Of 3330 
02429 
Oo .G 
Oey 325 
Oe20'6 2 
OR226 7 
One2002 
On 2692 
0.2404 
0.4504 
0.1244 
0.3126 
0.1189 
0.6094 


0.4754 


Factor Loading Matrix 


3 
0.3664 
0.4105 
0.3330 
0.3543 
0.1574 
0.1487 
0.3037 
0.5195. 
0.8000 
0.3170 
0.4168 
0.1459 
0.3269 
0.7469 
0.7425 
0.3571 
0.2694 


0.4018 


Pomc silicle wnderlying factor and could be more efficiently 
represented by a single dimension--i.e., by a single question 
Specne fitness report. AsSSigning an acceptable name to the 
factor requires both insight and judgment. An examination of 
the component variables reveals that they are principally 
task or goal oriented. They all measure some aspect of how 
an individual performs on the job. Since the sample popula- 
Prom cOnsSists entirely of officers, it can be assumed that a 
typical job for a member of the sample is more than routinely 
mechanical. It undoubtedly requires leadership capability, 
coupled with force, initiative, and judgment. Economy in 
Management can be regarded as a measure of effectiveness of 
Job performance and since nearly all Marine Officers have 
subordinates in their charge, the military concept of leader- 
Ship implies a training function in nearly all jobs. Thus, 
all the variables with very high loadings on the first factor 
can be regarded as elements of task orientation or accom- 
plishment. 

Poaditilonal insvont ianto the nature of the first factor 
can be gained from an examination of those variables which, 
euaemough not having "very high" factor loadings, nevertheless 
meacenigher loadings on this factor een on either of the 
other two. They are: 

1. Administrative Duties (variable 2) 

2. Yanedang Officers (variable 3) 

3. Handling Enlisted (variable 4) 

4. Attention to Duty (variable 8) 


5. General Value to the Service (variable 18) 
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Given the assumption that every Marine Officer inter- 
acts with both officer and enlisted subordinates in the per- 
formance of his duties, these variables are also task or goal 
oriented, with the possible exception of variable 18. De- 
ferring discussion of variable 18 for the moment, it seems 
Safe to assign to the first factor a name such as "Task 
Orientation and Effectiveness". 

Turning now to the second factor, the variables with 
very high factor loadings are: 

1. Personal Appearance (variable 6) 

2. Military Presence (variable 7). 
These variables relate to how an individual appears and how 
he behaves, both of which are external manifestations of an 
internal quality which may be called pride or self-image. 
It 1S interesting to note that the highest factor loadings 
of all the cases occurred here, for Personal Appearance on 
factor two. Apparently, then, the external manifestation is 
as important as the underlying self-image. In order to in- 
emude both aspects, and to eliminate any confusion with cur- 
Mently used names, the second factor will be called "Self- 
image and Bearing". 

The third and final factor had three variables with very 
high loadings. They are: 

1. Cooperation (variable 9) 
Ze loyalty (variable 14) 
3. Personal Relations (variable 15) 


MUS PedomeNcurdaot, Factor was Clearly task-oriented, the 
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third is clearly people-oriented. It measures interpersonal 
behavior, or interaction. In one sense it also measures team 
Spirit, since loyalty is presumably on a professional rather 
than personal level and applies to either the organization 

or its goals. The cooperation and personal relations, then, 
are with other members of the team. These variables measure 
more than just "Spirit", however. They also include a measure 
of how effectively the team spirit is established and main- 
tained. The third factor was therefore called "Team Orien- 
tation and Effectiveness". 

The three factors measured by a Marine Corps fitness 
report, then, relate principally to the task, the self, and 
the team. The three factors, using the names assigned above, 
are: 

1. Task Orientation and Effectiveness 

2. Self-image and Bearing 

3. Team Orientation and Effectiveness 
Two of these three factors correspond very closely to the 
results of a factor analysis conducted by Fleishman, Harris, 
and Burtt at Ohio State University (2). Their research was 
an attempt to isolate and define certain basic factors relat- 
ing to leadership and supervision in industry. They developed 
and administered a leadership description questionnaire which 
was then factor-analyzed. Their factor analysis produced two 
Gemmon factors, which they called "Initiating Structure" and 
"Interpersonal Effectiveness". The performance variables 


which factored into components of "Initiating Structure" 
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included those relating to patterns of task accomplishment. 
Eewola Git an this factor were the behavioral dimensions of 
leadership and aggressiveness (force). Their study did not 
identify a factor comparable to "Self-image and Bearing", 
however. Such factors would seem to be limited to organiza- 
tions for which external image or appearance is important--as 
Paechie military services or certain large corporations, but 
moe in industry (the environment for the Ohio State study). 

Pemceatea Garlier, the three conmem factors Of this 
Beroey collectively accounted for 73.5% of the total variance 
of all the variables. The equivalent information for any 
Specific variable is contained in its communality, which is 
the proportion of its variance accounted for by the common 
factors. The communality of each of the variables is shown 
in Table VI. 

Pesce DOolNt tt 16 possible to speculate about the 
Significance of variablé 18, General Value to the Service. 
If this variable actually measures what its terms and defi- 
nition indicate it should measure, it could indicate the 
relative importance of the three factors to the Marine Corps. 
Horeexample, its highest loading is on the first factor. 
This could, and probably does, indicate that Task Orientation 
and Effectiveness is the most important single dimension of 
Marine Corps performance, followed, though not too closely, 
fmaeene second and third factors in that order. This is not 
Empakbticularly surorising result, since the military community 


commonly stresses the primary importance of the mission (task). 
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Table VI. Communalities 


VARIABLE COMMUNALITY 
It Ue O59 
Z 0.6489 
S 0.6089 
4 O65954 
D Oro 219 
6 0.8706 
ti 0.8324 
8 OS IS 
5 Dmocie 7 

ao 02 /sa7 
ikdL 0.7747 
Le Gey 204 
Re 0.8274 
14 ; 0.6442 
1S; Oe 3155 
16 0.6614 
oy C7295 
18 0.8605 


As a final step in the analysis it was necessary to 
compare the relative scores of the 404 individuals on the 
three factors. This is the information that would be con- 
tained Limo Nessa GOUOte elt thewenree factors, rather than 
the 18 variables, were used. The table of factor scores in 
Appendix C, then, represents the fitness report data for the 
404 individuals in reduced form. A comparison of Appendices 
B and C reveals that those individuals with low scores on 
the first factor received low markings on one or more of the 
Variables of which that factor is composed. Similarly, high, 
factor scores equate to high markings. 

As an illustrative example, individual (case) number 367 
has a factor score of -5.05024 on the second factor. Appen- 
dix B reveals that this individual had markings of 1382 and 
1776 on the Personal Appearance and Military Presence var- 
iables, respectively. On the actual fitness reports for this 
individual, these numbers correspond to the following markings 
feerm Six reports): 

l. Personal Appearance: two Above Average, two Average, 
one Below Average and one Unsatisfactory. 

2. Military Presence: four Above Average and two Below 
Ee Gage . 

This is admittedly an extreme example, but it illustrates 
very clearly the correspondence between factor scores and 
actual markings. A thorough examination of Appendices B and 
C reveals that the correspondence between factor scores and 


fitness report markings holds throughout the set of data and 
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matcates that the three factors do, indeed, contain most of 


the essential information of the original 18 variables. 


Pee ANALYSIS OF SUBSETS OF THE DATA 


During this phase of the analysis, the original data 
was separated into the following eight subsets: one for each 
of the six ranks, one for Company Grade OF eeones and one for 
Field Grade Officers. The subsets were then analyzed by the 
Same procedures that were used for the complete set. Each 
subset factored into three factors, as before. The factor 
loadings, communalities, and factor scores changed, but the 
composition of the factors remained relatively constant 
throughout. In no case did the proportion of total variance 
accounted for by the three factors fall below 70%. In short, 
there seemed to be no significant difference in factor struc- 
ture between any of the groups (including the complete data 
set) studied. In view of this finding, it seemed neither 
necessary nor worthwhile to reproduce the tables for each of 
the subsets of data. 

One significant difference pe een the groups was, how- 
ever, revealed during this phase of the analysis. That dif- 
ference concerns the means of the markings on the various 
variables and is clearly visible in Table VII. The values 
aimed table are the arithmetic means of the values in Appen- 
dix B for the groups indicated. The last column of Table VIT, 
titled "Ali", is the mean of all values of the specified var- 


jable. The last row, titled "Group", shows the means for 
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each of the indicated groups for all variables. Two facts 
Stand out from this table: 

1. The mean marking for Second Lieutenant is higher 
than the corresponding marking for First Lieutenant in every 
case. 

2. The mean markings increase by rank from First Lieu- 
tenant in all but three cases, and the magnitude of the three 
reversals is very small. 

The trend indicated by paragraph (2) above is largely 
predictable, since it is assumed that only the "most quali- 
fied" individuals are promoted. The meaning of the reversal 
of this trend by First/Second Lieutenants is less clear, how- 
ever. One possible explanation is that Second Lieutenants, 
being largely young and inexperienced, are not judged by the 
Same standard as others-~i.e., not as much is expected of 
them. 

One other point from this table is worthy of mention. 
The mean marking for variable 14, Loyalty, is consistently 
higher than all others for each group and for the sample as 
a Whole. This indicates either that all Marine Officers are 
very loyal or that reporting officers are ill~-disposed to 
indicate otherwise. Considering the meaning which might be 
meeacned tO a low marking on this variable (e.g., disloyalty), 
the latter is undoubtedly the case. 

Table VIII shows the standard deviations of the markings 
by variable and by sub-group. The last column gives the 


standard deviation of all marks for the given variable, while 
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the last row gives the standard deviation for each of the 


sub-groups and is calculated as follows (11): 


a / 2 
Sm = M2 oF OM 


where, for each of the sub-groups, M is the mean of the 


G2 
variances of the variables and sé is the variance of their 
means. 

The main point to be learned from Table VIII is that 
the total dispersion of marks decreases as rank increases, 
although there is no clear trend for any specific variable. 
This means that the markings for higher ranks are not only 
better (higher) in the average than for lower ranks (with 
the exception pointed out above), but also that they are more 
consistent. It would not be true, however, to say that the 
variance (square of the standard deviation) decreases to the 
point where there is an insignificant difference between 
the marks for the higher ranks. Since the marks on the 
fitness reports are used for promotion and retention pur- 
poses, variance is essential for the proper execution of 
such decisions. It appears from the data that even though 
the variance decreases, it remains sufficient, even in the 


higher ranks, to satisfy the needs of the system by providing 


maeasis for selection. 


a2 





Vie Couch ton 


If personnel performance is important to an organiza- 
tion, it follows that performance evaluation is also important. 
It is not sufficient, however, simply to have a functioning 
system for the evaluation of performance. It is essential 
myae the system be both effective and efficient. The system 
Currently employed by the Marine Corps is apparently effective-- 
in the sense that it fulfills its objectives to the satisfac- 
tion of its users. The question raised and examined in this 
Study concerns not the effectiveness, but the efficiency of 
the system. Does the Marine Corps Fitness Report provide the 
necessary information in the most efficient way? Based on 
the analytic results presented earlier, it apparently does not. 

The 18 behavioral dimensions examined in this study 
Miidewotedly contain information which is vital to the needs 
of the Marine Corps, but nearly three-quarters of this in- 
formation could be acquired by asking only three questions, 
rather than 18, about the inoue toe being evaluated. As 
is frequently the case, there is a necessary trade-off be- 
tween comoleteness and efficiency. More information can be 
acquired by asking 18 questions than by asking three. And 
fee more Could be acquixed from 20, 30, or 50 questions. 

But the larger reports are undoubtedly less efficient, since 
there is a limit to the amount of useful and relevant infor- 


mation that can be acquired. 


The question to be answered, then, not by this paper 
but by the appropriate policy-makers, is: How much infor- 
mation is enough? If the amount of information contained 
me the three factors proposed by this study is not sufficient, 
then additional factors should be introduced. The additional 
factors should correspond to those variables with low factor 
loadings on the three common factors, since they are the 
ones least represented by the common factors. 

The ultimate decision in this matter, however, depends 
on the needs and objectives of the system. The three factors 
hypothesized by this study appear adequate to satisfy those 
objectives, as they have been previously defined. The three 
factors can provide an evaluation of duties performed; they 
can provide a comprehensive portrayal of the professional 
qualifications, personal traits and characteristics, and 
potential of the individual; and they can provide a basis 
for determining duty assignments and promotional qualifications. 
And they can do all this faster and more easily than is 


possible in the existing systen. 
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MATRIX OF MARKINGS 
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